Skip to content

chore(toolbox-core): Use Toolbox server dev build for integration tests #298

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
wants to merge 4 commits into
base: main
Choose a base branch
from
Open

chore(toolbox-core): Use Toolbox server dev build for integration tests #298

wants to merge 4 commits into from

Conversation

anubhav756
Copy link
Contributor

@anubhav756 anubhav756 commented Jul 1, 2025 •
edited
Loading

This PR updates the integration tests to use a dev build of the toolbox server from the main branch.

Currently, integration tests are conducted against a released version of the toolbox server. This poses a challenge when SDK and server features, such as optional parameters, are developed and released concurrently. In such scenarios, integration tests for the SDK will fail as the corresponding server-side support is absent in the released version.

To address this, this change modifies the server download for integration tests to pull from the GCS bucket for toolbox dev builds, specifically from the main branch. This ensures that the SDK is tested against the latest server version, which should ideally be stable and backward-compatible.

Flexibility in Versioning

For situations where new SDK features do not have a dependency on server-side changes, the TOOLBOX_VERSION and the newly introduced TOOLBOX_BUCKET env vars in the integration.cloudbuild.yaml config can be adjusted. This allows for switching back to a specific released version of the toolbox server for integration testing if needed.

Note

This PR removes the automatic v prefix from the version when constructing the blob download URL. Consequently, if you need to test against a released version (e.g., 0.8.0), the TOOLBOX_VERSION must be set to v0.8.0. This change accommodates version values that do not have a v prefix, such as main.

Note

The integration tests would currently fail due to a known issue with the optional parameters implementation with the server dev build [#771]

@anubhav756 anubhav756 self-assigned this Jul 1, 2025
@anubhav756 anubhav756 changed the base branch from main to anubhav-optional-params July 1, 2025 06:17
@anubhav756 anubhav756 changed the title Anubhav opt params test chore(toolbox-core): Use dev build of toolbox server for integration tests Jul 1, 2025
@anubhav756 anubhav756 marked this pull request as ready for review July 1, 2025 11:27
@anubhav756 anubhav756 requested a review from a team as a code owner July 1, 2025 11:27
@anubhav756 anubhav756 force-pushed the anubhav-opt-params-test branch from 8000adc to 3212d63 Compare July 1, 2025 13:30
@anubhav756 anubhav756 changed the title chore(toolbox-core): Use dev build of toolbox server for integration tests chore(toolbox-core): Use Toolbox server dev build for integration tests Jul 1, 2025
@anubhav756 anubhav756 force-pushed the anubhav-opt-params-test branch 3 times, most recently from fa6b4f4 to ca766a3 Compare July 2, 2025 13:43
@anubhav756 anubhav756 force-pushed the anubhav-optional-params branch 2 times, most recently from d6a153e to 57aeeb2 Compare July 2, 2025 14:22
@anubhav756 anubhav756 force-pushed the anubhav-opt-params-test branch from ca766a3 to 2ccb261 Compare July 2, 2025 14:30
@anubhav756 anubhav756 force-pushed the anubhav-optional-params branch 2 times, most recently from 1c598a4 to 90f7472 Compare July 2, 2025 16:17
@anubhav756 anubhav756 force-pushed the anubhav-opt-params-test branch from 2ccb261 to 9bee6e5 Compare July 2, 2025 16:18
@anubhav756 anubhav756 force-pushed the anubhav-optional-params branch from 90f7472 to 7e38892 Compare July 2, 2025 17:08
@anubhav756 anubhav756 force-pushed the anubhav-opt-params-test branch 3 times, most recently from c66ade4 to f811574 Compare July 2, 2025 17:21
kurtisvg
kurtisvg previously approved these changes Jul 2, 2025
View reviewed changes
Base automatically changed from anubhav-optional-params to main July 2, 2025 17:46
@anubhav756 anubhav756 dismissed kurtisvg’s stale review July 2, 2025 17:46

The base branch was changed.

@anubhav756 anubhav756 force-pushed the anubhav-opt-params-test branch 2 times, most recently from 3287116 to 1a0b5e7 Compare July 2, 2025 19:59
@anubhav756 anubhav756 requested a review from Yuan325 July 3, 2025 06:29
@twishabansal
Copy link
Contributor

How are the binaries added to the new bucket? Is this an automatic or a manual process?

@anubhav756
Copy link
Contributor Author

anubhav756 commented Jul 7, 2025 •
edited
Loading

How are the binaries added to the new bucket? Is this an automatic or a manual process?

AFAIK, the binaries are added to the new bucket through a manual process (correct me if I'm wrong @Yuan325)

I'm not entirely sure if we want to set up an automated process for creating binaries in the genai-toolbox-dev bucket just yet. For instance, do we want to update the binary as soon as a branch is merged? Or, should we treat this bucket as a "staging" area where we can release and test a new server binary with the SDK just before creating a public release.

I initially created this branch to test optional parameters feature, because it was unreleased at server's end as it needed to be released for both the server and the Python SDKs simultaneously, following an offline discussion. I've kept this PR open to accommodate any future requirements along these lines.

@kurtisvg Do you think we should set up a staging release process in the server repo?

@anubhav756 anubhav756 force-pushed the anubhav-opt-params-test branch from 1a0b5e7 to dcc035c Compare July 7, 2025 06:57
@anubhav756 anubhav756 requested a review from kurtisvg July 15, 2025 05:51
@anubhav756 anubhav756 force-pushed the anubhav-opt-params-test branch from 3787f40 to 1f0976b Compare July 15, 2025 05:51
@anubhav756 anubhav756 force-pushed the anubhav-opt-params-test branch from 1f0976b to 277bdb9 Compare July 21, 2025 07:14
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@twishabansal twishabansal Awaiting requested review from twishabansal

@Yuan325 Yuan325 Awaiting requested review from Yuan325

@kurtisvg kurtisvg Awaiting requested review from kurtisvg

At least 1 approving review is required to merge this pull request.

Assignees

@anubhav756 anubhav756

Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@anubhav756 @twishabansal @kurtisvg